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IMAGE PROCESSING 

This invention relates to an image processing or manipulation technique, and in 
particular to a technique that may be used to enhance our appreciation of 3D depth 
and form depicted in base 2D images and derivative virtual reality (VR) 
environments. 

When three-dimensional scenes are displayed or depicted using conventional 
two-dimensional techniques, for example by printing on to paper or the like or 
displaying on a television, computer monitor or the like, or by projection, although the 
brain perceives the images displayed as being to some extent three-dimensional in. 
nature, there are occasions when the perception of depth and form ia the displayed 
image is not particularly good. This is thought to be due to the absence of sujBacient 
monocular depth cues in the image to allow the brain to interpret the displayed image 
ftiUy. 

In the following description a distinction is made between 'pictures' and 
'images'. The following dejSnitions of pictures and images have been made by Prof 
Jan Koenderink (Vision Scientist), and the terms as used herein should be 
interpreted accordingly. 

Picture: "a real image, an ordered record, such as a video-signal, a photograph, an 
activity pattem in a neural structure. The order of the picture is purely conventional 
and exists only with respect to an extemal agent." 



Images: "always contain more than pictures because the whole structure of the 
receiver (a lifetime's experience) is summed up in them." 

This invention relates, at least to some degree, to the isolation of the 
techniques that transform pictures to images. 

The technique of the invention has been developed from tiheoretical work 
identifying a , new form of illusionaiy space which has been termed 'image space' 
(thus distinguishing itself from traditional picture space). Image space acknowledges 
the individual nature and specialisms of peripheral vision and central vision and also 
that brain function 'creates' important facets of visual perception by makiag 
'relativistic judgments' between the two. Key to the novelty of this approach is the 
realization that these new cues rely on as yet unappreciated monocular cues and the 
composition of monocular projection. These monocular cues distinguish this method 
of achieving apparent 3D depth from other techniques that rely on stereo cues 
deriving from binocular disparity etc. By correctiy representing and aligniag two 
projections used in monocular vision by the brain to compose mental images (the 
projection known as vision) it is possible to induce the brain to replicate depth and 
form perceptions that could otherwise only be created by directiy observing the 'real 
setting' from which the pictorial information derived. 



'Image Space' can be achieved/ induced in a number of ways pictoriaUy. 

1. By imbuing traditional picture space (perspective projection or 
photographic/ film record) witb the required cues. 

2. By using the principles to model a new iUusionary space to generate novel 
5 virtual reality (VR) environments. 

It is an object of the invention to provide a system, which may be used in 
combination Avith existing techniques, whereby an image may be created or modified 
to enhance the perception of the deptii in the original pictirre. In each case, image 
space relies on the incorporation of stimuU from which tiie brain is able to deduce 
3 additional perceptions of depth and form associated with our sense of visual 
perception. 

According to one aspect of tbe invention there is provided an image processing 
technique comprising the steps of: 

(a) stretching an original picture in the Y-direction by a factor falling 
witiun the range of 2-10%; 

selecting a fixation point and disordering the image centring the disordering 
operation around the fixation point; and 

rotating the image through an angle falling within the range 3-9°; 

(b) stretching a copy of the original image in tiie X-direction by a factor 
felhng within the range of 2-10%; and 
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selecting an area of the image around the selected fixation point; and 
(c) merging the selected area of the image formed in step (b) with the image 
formed in step (a). 

In step (a), the rotation of the image is preferably in a clockwise direction. A 
5 minority of individuals may prefer the hnage to be rotated in the opposite direction 
and which the technique is used to produce an image to be viewed by such 
individuals, the rotation may be undertaken in the anti-clockwise direction. 

The disordering operation preferably involves disordering in hne with a chosen 
self-similar firactal disorder pattern. 

10 . The image formed as a result of the tiransfoimation set out in step (a) wiU 

hereinafter be referred to as a Type 1 image, and that produced in the transformation 
set out in step (b) wiU be referred to as a Type 2 image. 

This technique can be thus be used to: 

i) mimic the appearance of the two base projections used by the brain in our 
15 monocularprojectionoftherealworldbytransformingknownpictiireenvironments. 

ii) repUcate a typical composition/ stiuctijre of these base projections used by 
the brain to form mental images. 

The key to forming a vaUd composition is the selection of a fixation point, the 
estabHshing of a fixation area (conforming to known constraints of central vision) and 
20 the correct imposition of conditions prevailing in 'the fixation volume', peripheral 



vision and around the interface between the Type 1 image and the Type 2 image. As 
in real time observation, a fixation point needs to be established on the media. The 
fixation point is the point aroimd which the various transformations techniques are to 
be centered. It is thought that the enhanced 3D depth and form cues are established 
by: 

1 . the brain as it makes relativistic judgments between interface between the 
two sets of information around central vision. 

2, an authentic representation of the specialist rendition of self-similar 
disordered information in peripheral vision providing a new form of spatial 
dimension. 

Without a central point of alignment (fixation point) firom both data sets, it 
would be impossible for the brain to make the required calculations. The fixation 
area establishes an area around the fixation point broadly analogous to the extent of 
central vision. As such, the fixation area establishes the extent of the image to be 
subjected to the specific conditions of the fixation volume. The fixation volxmie is a 
spherical area of influence contained within the fixation area. Any form (object), 
surface or part of a form that projects into this area of influence is projected in the 
brain as per the qualitative and quantitative characteristics of image type 2 above. 

It follbws that any other visual reference to objects outside of this fixation 
volume contained in the fixation area are referenced as per the image type 1 above. 



Areas of central vision outside the fixation volume are referenced as per the 
characteristics of 'peripheral vision'. 

There are a number of typical compositions used by the brain, which utilize 
aspects, and juxtapositions of the two image types in a way capable of generating the 
5 relativistic judgments from which the new cues are deduced. 

linages processed in this manner are thought to conform more fiiUy with our 
projection of real-time monocular observations (mental images) aUowing the brain to 
correctly interpret a fuUer range of three-dimensional cues from the 2D image or 
virtual reality setting. The projection of monocular vision would appear to be 
3 compiled from two data sets which aUows the brain to have certain degrees of 
freedom when compiling visual field. It has been noted that tiiese potential 
compositions have been adapted to suit the specific task being undertaken by the 
viewer. For example, reading 2D literatiire requires Htfle 3D infonnation. This order 
of infoimation could even provide a disti-action to the task in hand, hi this situation 
only one data set is used reducing the visual infonnation being processed. A speciaHst 
composition is used when detailed spatial analysis is requhed. One where half of 
centi-al vision is suppressed allowing for aspects of Type 1 mformation to spread into 
centiral vision up to the fixation pomt. This process allows the important brain 
function responsible for making relativistic judgements and hence enhances fonn cues 
to play over tiie actual object held in fixation. Aspects of this composition are often 
revealed in the work of visual artists painting still life material. 
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The nature and structure of information contained in peripheral vision is still 
largely unresolved by vision science. This invention nses the quaHties of self-similar 
(fractal) disorder (not blur) to repUcate the order of infonnation projected outside 
central vision (peripheral vision). This invention identifies a novel step in attributing 
5 peripheral vision with an independent 3D dimension centred on fixation, ie, the level 
of disorder generates out from fixation in every direction. Hence, 2D pictures are 
tiransfonned to replicate this order of information by building layers of disorder 
backwards, forwards and outward on the vertical plane from the targeted fixation. 
This estabhshes a new form of spatial dimension in tiie image. For stiU images this 
10 technique is seen to enhance depth perception in the image. However, the real 
advantage is tiiought to he in its abiHty to impart accurate renditions of perceived 
movement in moving images. Once mastered, movement around tiie targeted fixation 
will now be closer to our perception of movement observed from 'real settings'. 
Moving images will also require changes to the targeted fixation points as the 
15 action depicted on them develops. Techniques for inducing a change in fixation wiU 
be incorporated into action depicted in moving media to lead the viewer tirrough the 
eye movement procedure. . 

The processing technique may require additional steps of fine-tuning around 
the boundary between the images formed in steps (a) and (b). 
20 Step (a) of the processing technique may fiirther include steps of altering the 

conti-astof Type 1 informationbyafactorfalhng within the range of +10-40% and/or 
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decreasing the brightness of the image by a factor falling within the range 2-40%. If 
contrast is increased in this way, then colom- saturation should preferably be reduced 
to an accommodation level (usually to match the increase of level in contrast). 

The technique dejBned hereinbefore may be performed using a computer on 
image data derived, for example, from a scanner or digital camera, or on digital 
picture data obtained or created from other sources including computer programs. 
However, it will be appreciated that the technique may be performed ia other ways, 
and the invention is not restricted to the specific arrangements or applications 
described hereia. 

The invention wiU fiirther be described, by way of example, with reference to 
the accompanying drawings, in which: 

Figure 1 is an illustration prior to use of the image processing technique; 

Figures 2 and 3 illustrate stqps of the image processing techniques; 

Figures 4 and 5 illustrate the processed form of the illustration of Figure 1; 

Figures 6 to 10 are ftirther views illustrating the effect of the process; 

Figure 1 1 is a diagram illustrating part of the process; and 

Figure 12 is a diagram illustrating a modification to the technique. 

The invention will, initially, be described with reference to the image 
iUustrated in Figure 1. As can clearly be seen. Figure 1 is a computer generated or 
virtual reality representation of a vase 10, a cone 12, spheres 14 and cubes 16. As 
each of the objects illustrated are three-dimensional objects, the illustration uses 



isometric projections to provide depth cues to be interpreted by tiie brain in an attempt 
to make the objects appear to be an accurate representation of their form in three- 
dimensional space. Each object is isolated from all of the other objects, and no 
central perspective cues to indicate the relative positions of the objects are present 
As a result, it is difficult for the brain to interpret the image to determine, for 
example, the relative sizes of the objects and their positions in three dimensional 
space. In the image, both of the cubes are drawn as the same size and are perceived 
as beiBig the same size despite one being fiirfiier away than the other. 
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In accordance with liie invention, the base image of Figure 1 is processed to 
form an image of improved fonn. This is achieved by taking the image of Figure 1 
and performing a transformation thereon whereby tiie image is stretched in the Y- 
direction (vertical) by a factor faUing within the range 2- 10%, preferably about 5%, is 
disordered and is rotated through an angle falling within the range 3-9° clockwise, 
preferably 6° clockwise. Figure 2 illustrates the effect of the stretching and rotation 
of the image, but (for clarity) not the impact ©rdisordering the image. The order in 
vMch these operations may be varied, but the values/factors used may need to be 
varied if the order is not as described. 

Another copy of the image is stretched by a factor of 2-10% in the X-direction 
(horizontal). Preferably, it is stretched to ensure a differential with the image in the 
previous transfoimation of around 5-20%. A fixation point on the image is selected, 
and an area around the fixation point is selected, the remainder of the image being 
deleted. The result of this operation is shown in Figure 3, where the centre of the 
vase has been selected as the fixation point. 

The two images produced in this manner are superimposed upon one another. 
The images may be superimposed using several different techniques. Figure 4 
illustrates the effect of taking the image of Figure 2 and simply replacing an area 
around the fixation point with the image of Figure 3 . Figure 5 illustrates the result of 
an alternative superimposition technique wherein the two images are overlaid and 
parts of both images are visible, the vase taldng the form of a fused form. The 
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fixation area is the vase; this comprises a combined form fosed from both image types 
1 and2inwhatatfirst seems anodd looking conjunction. However, when the centre 
of the vase is fixated, the two identities are merged by die brain into a single percept. 
This percept is of a more three dimensional vase. This capacity for increased 3D 
5 percept is now extended to all the olber objects in the picture. For example the 
fimher of the two cubes is now perceived to be larger than the closer cube. 

Refening next to Figures 6a, 6b and 6c, Figure 6a shows in schematic form a 
bottle 20 located between six verticaUy extending bars 22. The illustration of Figure 
6a is modified or processed in accordance with an embodiment of the invention to 

10 form Ihe image of Figure 6b. For clarity, agam, the disordering operation has been 
omitted. When the image of Figure 6b is viewed by fixating upon &e center of the 
botde 20, an improved degree of depth can be seen in the image as compared to the 
picture of Figure 6a, the image of Figure 6b containing a greater level of depth cues 
forthebraintointeipretthatdieillustrationshowninFigure6a. Figure 6c is another 

15 iinageformedusingtheprocessdescribedhereinbefore,buti^ingadifferent;smaUer 

part of the original illustration as the fixation area. 

Figures 7a, 7b and 7c are similar to Figures 6a, 6b and 6c but illustrating a 
bottle 24 positioned upon a table 26. 

This percept of 3D form and depth is, in some ways, analogous to the percept 
20 obtained by perspective space. However, it is now thought that perspective cues are 
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not the principle method by which spatial depth is obtained in central vision or 
through the fiiU extent of visual field. Rather, we see spatial depth and fonn by a 
process that involves our visual system making relativistic judgments between two 
distinct data sets or image types. The arrangement identified above stimulates the 
brain to engage with this process when observing 2D space, providing us with cues 
more conversant with our appreciation of real settings. 

The new image space allows us to create depth in a similar way to the way it is 
perceived in our actual visual appreciation of our surroundings. Looking at the new 
image space will be closer to an actual act of real time observation. 

Referring next to Figures 8 to 10, an original digitised photograph of a real 
setting is shown in Figm-e 8. The key to forming a vaHd composition is the selection 
of a fixation point and fixation area, and the correct imposition of conditions 
prevailing in 'the fixation volume' aad peripheral vision. In the present case, the 
centre of the bottie has been selected as the fixation point Figure 9 shows the effect 
of performing the transformations described hereinbefore, but omitting the disordering 
operation for clarity, and simply replacing the part of the image around the fixation 
point on one image with the relevant part of the other image. Items within tiie 
fixation area, established round the fixation point, are represented as image type 2 
with some of the associated quahtative and quantitative transformations. Outside the 
fixation area, all objects and surfaces are rendered as per image type 1 with the 
associated qualitative and quantitative transformations. As can be seen in Figure 9, 
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the boundaiy 18 between the two images in the merged image is clearly visible. 
Figure 10 illustrates the result of a modified superimposition technique which 
removes or reduces tlie impact of the boundary between the images^ and shows the 
establishment of the fixation volume where all surfaces and objects not contained by 
5 the spherical influence of the fixation volume are also rendered as per image type 1, 

For example; around the neck of the bottle the stones of the waU, although 
within the circumference of the fixation area of Figure 9, are too far firom the bottle to 
be included in the fixation volume and so are referenced as per image Type 1. 

For example; one arm of the candelabra penetrates the fixation volume and so 
10 is referenced as per image Type 2 while the rest of the object is outside this area of 
influence and so is rendered as per image type 1. 

By comparing Figures 9 and 10 with Figure 8 it is apparent that, in addition to 
the transformations mentioned hereinbefore, the image which has been stretched in 
the Y-direction has also been modified by increasing the level of contrast and 
15 reducing the level of colour saturation. Also the brightness of the image has been 
reduced. The contrast has been increased by an amount falling within the range 10- 
40%, preferably about 25%, and the brightness has been reduced by a factor falling 
within the range 2-40%, preferably about 10%. The toning down of colour saturation 
is to balance for the effects of higher contrast and is broadly proportional, 

20 It is thought that images processed using the techniques described hereinbefore 
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are of improved or increased resemblance to mental images projected by the brain. 
Using this structure/ armature in digitised medial, it will be possible to enter many 
facets of the differences known to exist between peripheral vision and central vision. 
Clear and accurate representation of mental image compositions allows an observer of 
the 2D rendition to engage in additional computations leading to a more viable 
perception of the real setting being depicted. 

An observer of the transformed image will need to look at (fixate on) the 
selected fixation point to perceive/ appreciate Hie fiill impact of the additional cues. 

In the description hereinbefore reference has been made to a step of 
disordering the image. Although a number of disordering techiiiques may be used, a 
preferred technique is to incrementally disorder the image using, for example, the 
sunflower-like fi-actal pattem shown in Figure 11 to reduce the amount of detail 
visible in the processed image with increasing distance firom the fixation point. The 
novel aspect iacluded as part of this submission refers to the use of and adaptation of, 
this pattem into a 3D spatial dimension independent to other 3D cues used in picture 
information (perspective for example). 

Figure 12 is a diagrammatic view illustrating one way in which the disordering 
operation may, preferably, be performed. The image of Figure 12 includes, at its 
fixation point, a vase 30. Three flat vertical screens 32 are positioned, in line, in front 
of and to one side of the vase 30. A tree branch 34 extends across the foreground, in 
front of aU of the other objects. In the arrangements described hereinbefore the 
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disordering operation has been conducted so as to introduce only a small level of 
disorder into the part of the image immediately surrounding the fixation point, the 
degree of disorder increasing, in line with the sunflower pattern of Figure 1 1, with 
iacreasing distance from the fixation point. In the arrangement of Figure 12, although 

5 this general approach has been taken, the disordering operation has been modified to 
identify the boundaries or edges of the objects illustrated and to introduce a disruption 
or jump in the degree of disordering at the edges or boundaries. The disruptions to 
the degree of disorder give rise to fiuther spatial depth cues in the image for 
interpretation by the viewer's brain. 

10 It is thought that images processed using the techniques described hereinbefore 

provide an improved or increased perception or mental projection of depth to the part 
of the image around the fixation point and through peripheral areas. The enhanced 
perception of depth is apparent both where a viewer views the image through both 
eyes and when the image is viewed using only one eye. Stereo blind subjects perceive 

15 the additional depth these transformed images contain. In contrast, the organised 
depth of stereo based assemblages fail to engage stereo blind individuals. 

Although the techniques have been described hereinbefore in relation to the 
processing of pictures, either in the form of computer generated pictures or 
photographs, it will be appreciated that this need not be the case and the invention 

20 may be applied to a wide range of technologies and display techniques including 
computer or television displays, projected images, virtual reality displays, animations 
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and printed media. The invention may also find application in, for example, cameras 
(either for stills or for moving images) which are designed to incorporate a display on 
which the transformed image can be displayed in "real time" so that the photographer, 
director or other person involved in the capturing of the image can see how it will be 
5 transformed before the image is captured, or completion of the capture takes place, hi 
order to accommodate this, the camera may be provided sensors to allow the depths of 
objects in the j&eld of vision to be measured at the same time as the image is captured, 
these two components being used in conjmiction with one another and in conjunction 
with the techniques described hereinbefore in the transformation of the image. 
10 Additionally, the location of the fixation point in the field of view will need to be 
input. This could be set at the centre of the field of view or, using appropriate input 
means, may be selected by the director, photographer or other individual. 



( 
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CLAIMS 

1 . An image processing technique comprising the steps of: 

(a) stretching an original image in the Y-direction (vertical) by a factor 
5 falling within the range of 2- 1 0%; 

selecting a fixation point and disordering the image centring the disordering 
operation around the fixation point; and 

rotating the image through an angle falling within the range 3-9°; 

(b) stretching a copy of the original image in the X-direction (horizontal) by 
10 a factor falling within the range of 2- 10%; and 

selecting an area of the image aroimd the selected fixation point; and 

(c) merging the selected area of the image formed in step (b) with the image 
formed in step (a). 

15 2, A technique as claimed in Claim 1, finther comprising an additional 

step of fine -tuning the boimdary between the images formed in steps (a) and (b). 

3. A technique as claimed in Claim 1 or Claim 2, wherein step (a) of the 
processing technique further includes steps of altering the contrast by a factor falling 
20 within the range of 10-40% and/or decreasing the colour saturation of the image by a 
factor falling within the range 10-40%. 
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4. A technique according to Claim 3, furfher comprising a step of 
decreasing the brightness by a factor felling within the range 2-40%. 

5. A technique according to any one of the preceding claims, wherein the 
rotation is undertaken in the clockwise direction. 

6. A technique according to any one of the preceding claims, wherein the 
disordering operation of step (a) involves disordering the image in line with a self- 
similar fractal disorder pattern. 

7. A technique according to any one of the preceding claims, furfher 
comprising, in step (a), identifying at least one boundary or edge of at least one object 
and introducing a disruption in the degree of disordering at the said boundaiy or edge. 

8. A computer programmed to perfonn the technique of any of the 
preceding claims on image data derived jfrom a scanner, a digital camera, or on digital 
image data obtained or created jfrom other sources including computer programs. 

9. An image processing technique substantially as hereinbefore described 
with reference to the accompanying drawings. 
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ABSTRACT 



IMAGE PROCESSING 

An image processing technique comprises fhe steps of: 

(a) stretching an original image in the Y-direction (vertical) by a factor 
falling within the range of 2-10%; 

selecting a fixation point and disordering the image centring the disordering 
operation aroimd the fixation point; and 

rotating the image through an angle falling within the range 3-9°, preferably 
clockwise; 

(b) stretching a copy of the original image inihe X-direction (horizontal) by 
a factor falling within the range of 2-10%; and 

selecting an area of the image around the selected fixation point; and 

(c) mergiiigr,th.e selected area of the image formed in step (b) with the 
■ ■ Mage formed in step (a). 




